Programmable Analog Pulse-Firing Neural Networks

Lionel Tarassenko Dept. of Eng. Science, University of Oxford, Parks Road, Oxford, OX1 3PJ United Kingdom. We describe pulse stream firing integrated circuits that implement asynchronous analog neural networks. Synaptic weights are stored dynamically, and weighting uses time-division of the neural pulses from a signalling neuron to a receiving neuron. MOS transistors in their "ON" state act as variable resistors to control a capacitive discharge, and time-division is thus achieved by a small synapse circuit cell. The VLSI chip set design uses 2.5J.1.m CMOS technology. INTRODUCTION Neural network implementations fall into two broad classes digital [1,2] and analog (e.g. [3,4]). The strengths of a digital approach include the ability to use well-proven design techniques, high noise immunity, and the ability to implement programmable networks. However digital circuits are synchronous, while biological neural networks are asynchronous. Furthermore, digital multipliers occupy large areas of silicon. Analog networks offer asynchronous behaviour, smooth neural activation and (potentially) small circuit elements. On the debit side, however, noise immunity is low, arbitrary high precision is not possible; and no reliable "mainstream" analog nonvolatile memory technology exists. Many analog VLSI implementations are nonprogrammable, and therefore have fixed functionality. For instance, subthreshold MOS devices have been used to mimic the nonlinearities of neural behaviour, in implementing Hopfield style nets [3] , associative memory [5] , visual processing functions [6] , and auditory processing [7]. Electron-beam programmable resistive interconnects have been used to represent synaptic weights between more conventional operational-amplifier neurons [8,4]. We describe programmable analog pulse-firillg neural networks that use 00chip dynamic analog storage capacitors to store synaptic weights, currently 671 672 Hamilton, Murray and Tarassenko refreshed from an external RAM via a Digital -Analog converter. PULSE-FIRING NEURAL NETWORKS A pulse-firing neuron, i is a circuit which signals its state, V. by generating a stream of 0-5V pulses on its output. The pulse rate R.' varies from 0 when neuron i is OFF to R.(max) when neuron i is fully ION. Switching between the OFF and ON stAtes is a smooth transition in output pulse rate between these lower and upper limits. In a previous system, outlined below, the synapse allows a proportion of complete presynaptic neural pulses V. to be added (electrically OR-ed) to its output. A synaptic "gating" function, determined by T .. , allowed bursts of complete pulses through the synapse. Moving down a'l column of synapses, therefore, we see an ever more crowded asynchronous mass of pulses, representing the aggregated activity of the receiving neuron. In the system that forms the substance of this paper, a proportion (determined by T .. ) of each presynaptic pulse is passed to the postsynaptic summation. l] INTEGRATOR RING OSCLLATOR ~------------------------~I~I --------------------------------~ 11111111111111111111111111 Excitatory "-.... A~ ~ 11111111111111111111111111